Disable gzip compression for float NIfTI intermediates by nx10 · Pull Request #285 · childmindresearch/rbc

nx10 · 2026-04-09T01:58:25Z

Summary

Write intermediate float volumes as uncompressed .nii instead of .nii.gz to avoid gzip overhead on data that is read back immediately
Replace fslsplit with nibabel four_to_three in split_4d(), which also removes a container call
Integer and binary masks are left compressed since they compress efficiently

Changed intermediates

Location	Intermediate	Why
`common.py:split_4d()`	per-volume splits	Hundreds of float 3D volumes in a tight loop (biggest win)
`resampling.py`	`vol__motion.nii`, `vol__template.nii`	Per-volume ANTs outputs merged immediately
`longitudinal/transform.py`	`vol_*_template.nii`	Same pattern as resampling
`motion.py`	`slice.nii`	Float BOLD reference, read once by 3dvolreg
`masking.py`	`probseg_transform`, `bold_ref_dir_corrected`, `ref_bold_corrected`, `ref_bold_corrected_brain_masked`, `uni`	Float intermediates in chained masking pipeline
`distortion.py`	`phasediff`, `merged_epi`	Float intermediates consumed immediately

Test plan

Unit tests pass (725/725)
Ruff + mypy clean
Compare full pipeline CI duration against recent main runs

Closes #252

Write intermediate float volumes as uncompressed .nii instead of .nii.gz to avoid gzip overhead on data that is read back immediately. Integer and binary masks are left compressed since they compress efficiently. - Replace fslsplit with nibabel four_to_three in split_4d (also removes a container call) - Use .nii for per-volume ANTs outputs in resampling and longitudinal transform loops - Use .nii for motion reference slice, masking float intermediates, and distortion correction float intermediates Closes #252

github-actions · 2026-04-09T01:59:08Z

Coverage Report

File	Stmts	Miss	Cover	Missing
rbc
__init__.py	1	0	100%
context.py	25	8	68%	70, 75–77, 79–80, 93–94
metadata.py	56	0	100%
rbc/bids
__init__.py	9	0	100%
_schema.py	585	4	99%	776, 782, 790, 1101
anatomical.py	24	3	87%	44, 47–48
builder.py	72	7	90%	233–235, 362, 364–365, 386
functional.py	42	5	88%	46–49, 84
longitudinal.py	25	0	100%
metrics.py	23	1	95%	44
qc.py	17	0	100%
query.py	67	42	37%	103–107, 121–125, 127–128, 130–135, 137, 139, 153, 159–165, 198, 207, 209–216, 225, 257, 266–267
session.py	47	0	100%
rbc/cli
__init__.py	1	0	100%
all.py	49	2	95%	84, 108
anatomical.py	25	2	92%	47, 62
base.py	67	8	88%	55, 115–117, 123–125, 148
functional.py	33	2	93%	67, 86
longitudinal.py	27	2	92%	46, 62
main.py	41	0	100%
metrics.py	42	2	95%	77, 94
qc.py	25	2	92%	45, 60
rbc/core
__init__.py	3	0	100%
common.py	26	12	53%	43–45, 62–70
fileops.py	27	4	85%	69–72
fsl2itk.py	42	0	100%
nifti.py	192	5	97%	236–237, 244–245, 524
niwrap.py	56	1	98%	58
rbc/core/anatomical
__init__.py	4	0	100%
registration.py	15	4	73%	59, 151, 166, 183
segmentation.py	24	8	66%	64, 74–76, 92, 114, 125, 141
rbc/core/functional
__init__.py	13	0	100%
coregistration.py	7	2	71%	44, 55
despiking.py	7	3	57%	32, 36–37
distortion.py	130	40	69%	269–271, 321, 324, 332–335, 341–346, 349, 352–353, 356, 365–369, 375–376, 387–388, 391, 397, 443–444, 447, 455, 461, 470–471, 474, 484, 491
erosion.py	32	1	96%	50
initialization.py	9	4	55%	35, 42–43, 63
masking.py	34	25	26%	53, 55–56, 58–59, 62–65, 69, 91, 134, 183, 197, 208, 223, 233, 249, 258, 271, 285, 296, 306, 319, 328
motion.py	57	37	35%	62, 64–67, 69, 71, 73, 76–77, 83–84, 86–87, 95–97, 99, 102, 105, 107, 124–125, 135–138, 159, 169, 171–172, 175, 177–179, 181, 183
nuisance.py	81	60	25%	78, 80–85, 87, 89–90, 93, 96–98, 100–102, 104–110, 112–116, 118, 163, 165, 167, 170–171, 173–175, 178, 181–182, 185–187, 190–193, 197, 203–205, 207, 235, 243, 269, 278, 307, 316, 322
regressors.py	89	6	93%	163, 193, 325–328
resampling.py	54	43	20%	37–42, 74–76, 78, 80–81, 85–87, 90, 93, 105, 107–108, 112, 114, 150–152, 154–155, 159, 161–162, 167–169, 173–174, 177, 180, 187, 199, 201–202, 207, 209
timing.py	16	11	31%	46–47, 49–53, 58, 60, 66–67
rbc/core/longitudinal
__init__.py	1	0	100%
transform.py	46	7	84%	106–107, 165–168, 170
rbc/core/metrics
__init__.py	3	0	100%
alff.py	90	1	98%	265
reho.py	66	0	100%
smoothing.py	7	3	57%	36, 42–43
standardization.py	27	11	59%	64, 66–68, 70, 72–75, 77–78
timeseries.py	57	1	98%	120
rbc/core/qc
__init__.py	6	0	100%
dvars.py	26	0	100%
motion.py	31	0	100%
registration.py	41	0	100%
xcp.py	41	0	100%
rbc/orchestration
__init__.py	31	2	93%	88, 91
all.py	41	23	43%	76–78, 80–81, 85, 92, 95, 100, 103, 111, 121, 123–124, 129, 137, 145–146, 163, 165–166, 168, 170
anatomical.py	37	22	40%	49–52, 57–58, 60–63, 85–87, 89–90, 94, 101, 104, 109–110, 116, 118
functional.py	39	23	41%	60–62, 64–65, 67, 69, 84–86, 88, 116–118, 120–121, 125, 131, 134, 139–140, 149, 151
longitudinal.py	56	1	98%	150
metrics.py	46	30	34%	32–35, 38–40, 67, 75–76, 100–102, 104–106, 110, 118–121, 123–124, 130–132, 137, 145, 152, 154
qc.py	34	0	100%
rbc/workflows
__init__.py	10	0	100%
anatomical.py	47	19	59%	90–93, 97–102, 107, 177–180, 182–184, 188
functional.py	98	56	42%	113–114, 122, 127–128, 136, 207–208, 211–212, 215–216, 219, 222–225, 235–237, 247–248, 251, 254, 257–258, 266–267, 274–275, 282–283, 290–291, 294–296, 299–302, 314–315, 327, 329–332, 335–336, 345–346, 354, 361, 430, 436
metrics.py	39	17	56%	80–81, 86–87, 90–93, 96–99, 103–106, 108
qc.py	53	33	37%	87, 89–90, 93, 96, 99–104, 107, 116–118, 122–125, 127–131, 133–134, 136–138, 141, 158, 165, 167
rbc_resources
__init__.py	38	0	100%
TOTAL	3232	605	81%

Tests	Skipped	Failures	Errors	Time
726	0 💤	0 ❌	0 🔥	10.075s ⏱️

FSL uses FSLOUTPUTTYPE to determine the output format, ignoring the filename extension. Keep fslmaths outputs as .nii.gz since the Docker container defaults to NIFTI_GZ.

kaitj

In principle, I think I agree with this. Do we have a general idea of how much of a difference there is between the compressed and uncompressed file sizes?

nx10 · 2026-04-09T14:00:48Z

CI reported intermediate dir size 6.3GB before and after these changes so <100MB - generally I picked non/hard compressible float array niftis so this is not surprising

Revert fslmaths output to .nii.gz (FSL ignores extension in Docker)

9da1d30

FSL uses FSLOUTPUTTYPE to determine the output format, ignoring the filename extension. Keep fslmaths outputs as .nii.gz since the Docker container defaults to NIFTI_GZ.

nx10 mentioned this pull request Apr 9, 2026

Replace fslmaths calls with nibabel + numpy/scipy #286

Open

nx10 marked this pull request as ready for review April 9, 2026 13:52

nx10 requested review from jpillai00 and kaitj April 9, 2026 13:52

kaitj approved these changes Apr 9, 2026

View reviewed changes

nx10 merged commit eccd701 into main Apr 9, 2026
8 checks passed

nx10 deleted the feat/uncompressed-nifti-intermediates branch April 9, 2026 14:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable gzip compression for float NIfTI intermediates#285

Disable gzip compression for float NIfTI intermediates#285
nx10 merged 2 commits into
mainfrom
feat/uncompressed-nifti-intermediates

nx10 commented Apr 9, 2026

Uh oh!

github-actions Bot commented Apr 9, 2026 •

edited

Loading

Uh oh!

kaitj left a comment

Uh oh!

nx10 commented Apr 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nx10 commented Apr 9, 2026

Summary

Changed intermediates

Test plan

Uh oh!

github-actions Bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kaitj left a comment

Choose a reason for hiding this comment

Uh oh!

nx10 commented Apr 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Apr 9, 2026 •

edited

Loading